Meta-analysis of Placebo

Mean effect sizes for changes in depression were calculated for 2,318 patients who had been randomly assigned to either antidepressant medication or placebo in 19 double-blind clinical trials. As a proportion of the drug response, the placebo response was constant across different types of medication (75%), and the correlation between placebo effect and drug effect was .90. These data indicate that virtually all of the variation in drug effect size was due to the placebo characteristics of the studies. The effect size for active medications that are not regarded to be antidepressants was as large as that for those classified as antidepressants, and in both cases, the inactive placebos produced improvement that was 75% of the effect of the active drug. These data raise the possibility that the apparent drug effect (25% of the drug response) is actually an active placebo effect. Examination of pre–post effect sizes among depressed individuals assigned to no-treatment or wait-list control groups suggest that approximately one quarter of the drug response is due to the administration of an active medication, one half is a placebo effect, and the remaining quarter is due to other nonspecific factors. EDITORS' NOTE The article that follows is a controversial one. It reaches a controversial conclusion—that much of the therapeutic benefit of antidepressant medications actually derives from placebo responding. The article reaches this conclusion by utilizing a controversial statistical approach—meta-analysis. And it employs meta-analysis controversially—by meta-analyzing studies that are very heterogeneous in subject selection criteria, treatments employed, and statistical methods used. Nonetheless, we have chosen to publish the article. We have done so because a number of the colleagues who originally reviewed the manuscript believed it had considerable merit, even while they recognized the clearly contentious conclusions it reached and the clearly arguable statistical methods it employed. We are convinced that one of the principal aims of an electronic journal ought to be to bring our readers information on a variety of current topics in prevention and treatment, even though much of it will be subject to heated differences of opinion about worth and ultimate significance. This is to be expected, of course, when one is publishing material at the cutting-edge, in a cutting-edge medium. We also believe, however, that soliciting expert commentary to accompany particularly controversial articles facilitates the fullest possible airing of the issues most germane to appreciating both the strengths and the weaknesses of target articles. In the same vein, we welcome comments on the article from readers as well, though for obvious META-ANALYSIS OF PLACEBO http://journals.apa.org/pt/prevention/volume1/pre0010002a.html 1 of 16 3/11/2009 2:51 PM reasons, we cannot promise to publish all of them. Feel free to submit a comment by emailing [email protected]. Peter Nathan, Associate Editor (Treatment) Martin E. P. Seligman, Editor We thank R. B. Lydiard and Smith-Kline Beecham Pharaceuticals for supplying additional data. We thank David Kenny for his assistance with the statistical analyses. We thank Roger P. Greenberg and Daniel E. Moerman for their helpful comments on earlier versions of this paper. Correspondence concerning this article should be addressed to Irving Kirsch, Department of Psychology, U-20, University of Connecticut, 406 Babbidge Road Storrs, CT 06269-1020. E-mail: [email protected] More placebos have been administered to research participants than any single experimental drug. Thus, one would expect sufficient data to have accumulated for the acquisition of substantial knowledge of the parameters of placebo effects. However, although almost everyone controls for placebo effects, almost no one evaluates them. With this in mind, we set about the task of using meta-analytic procedures for evaluating the magnitude of the placebo response to antidepressant medication. Meta-analysis provides a means of mathematically combining results from different studies, even when these studies have used different measures to assess the dependent variable. Most often, this is done by using the statistic d, which is a standardized difference score. This effect size is generally calculated as the mean of the experimental group minus the mean of the control group, divided by the pooled standard deviation. Less frequently, the mean difference is divided by standard deviation of the control group (Smith, Glass, & Miller, 1980). Ideally, to calculate the effect size of placebos, we would want to subtract the effects of a no-placebo control group. However, placebos are used as controls against which the effects of physical interventions can be gauged. It is rare for an experimental condition to be included against which the effects of the placebo can be evaluated. To circumvent this problem, we decided to calculate within-cell or pre–post effect sizes, which are the posttreatment mean depression score minus the pretreatment mean depression score, divided by the pooled standard deviation (cf. Smith et al., 1980). By doing this for both placebo groups and medication groups, we can estimate the proportion of the response to antidepressant medication that is duplicated by placebo administration, a response that would be due to such factors as expectancy for improvement and the natural course of the disorder (i.e., spontaneous remission). Later in this article, we also separate expectancy from natural history and provide estimates of each of these effects. Although our approach is unusual, in most cases it should provide results that are comparable to conventional methods. If there are no significant pretreatment differences between the treatment and control groups, then the subtraction of mean standardized pre–post difference scores should result in a mean effect size that is just about the same as that produced by subtracting mean standardized posttreatment scores. Suppose, for example, we have a study with the data displayed in Table 1. The conventionally calculated effect size would be would be 1.00. The pre–post effect sizes would be 3.00 for the treatment group and 2.00 for the control group. The difference between them is 1.00, which is exactly the same effect calculated from posttreatment scores alone. However, calculating the effect size in this manner also provides us with the information that the effect of the control procedure was 2/3 that of the treatment procedure, information that we do not have when we only consider posttreatment scores. Of course, it is rare for two groups to have identical mean pretreatment scores, and to the extent that those scores are different, our two methods of calculation would provide different results. However, by controlling for baseline differences, our method should provide the more accurate estimate of differential outcome. META-ANALYSIS OF PLACEBO http://journals.apa.org/pt/prevention/volume1/pre0010002a.html 2 of 16 3/11/2009 2:51 PM Table 1 Hypothetical Means and Standard Deviations for a Treatment Group and a Control Group Treatment Control Pretreatment Posttreatment Pretreatment Posttreatment M 25.00 10.00 25.00 15.00 SD 5.50 4.50 4.50 5.50 The Effects of Medication and Placebo Study Characteristics Studies assessing the efficacy of antidepressant medication were obtained through previous reviews (Davis, Janicak, & Bruninga, 1987; Free & Oei, 1989; Greenberg & Fisher, 1989; Greenberg, Bornstein, Greenberg, & Fisher, 1992; Workman & Short, 1993), supplemented by a computer search of PsycLit and MEDLINE databases from 1974 to 1995 using the search terms drug-therapy or pharmacotherapy or psychotherapy or placebo and depression or affective disorders. Psychotherapy was included as a search term for the purpose of obtaining articles that would allow estimation of changes occurring in no-treatment and wait-list control groups, a topic to which we return later in this article. Approximately 1,500 publications were produced by this literature search. These were examined by the second author, and those meeting the following criteria were included in the meta-analysis: The sample was restricted to patients with a primary diagnosis of depression. Studies were excluded if participants were selected because of other criteria (eating disorders, substance abuse, physical disabilities or chronic medical conditions), as were studies in which the description of the patient population was vague (e.g., "neurotic"). 1. Sufficient data were reported or obtainable to calculate within-condition effect sizes. This resulted in the exclusion of studies for which neither pre–post statistical tests nor pretreatment means were available. 2. Data were reported for a placebo control group. 3. Participants were assigned to experimental conditions randomly. 4. Participants were between the ages of 18 and 75. 5. Of the approximately 1,500 studies examined, 20 met the inclusion criteria. Of these, all but one were studies of the acute phase of therapy, with treatment durations ranging from 1 to 20 weeks (M = 4.82). The one exception (Doogan & Caillard, 1992) was a maintenance study, with a duration of treatment of 44 weeks. Because of this difference, Doogan and Caillard's study was excluded from the meta-analysis. Thus, the analysis was conducted on 19 studies containing 2,318 participants, of whom 1,460 received medication and 858 received placebo. Medications studied were amitriptyline, amylobarbitone, fluoxetine, imipramine, paroxetine, isocarboxazid, trazodone, lithium, liothyronine, adinazolam, amoxapine, phenelzine, venlafaxine, maprotiline, tranylcypromine, and bupropion. The Calculation of Effect Sizes In most cases, effect sizes (d) were calculated for measures of depression as the mean posttreatment score minus the mean pretreatment score, divided by the pooled standard deviation (SD). Pretreatment SDs were used in place of pooled SDs in calculating effect sizes for four studies in which posttreatment SDs were not reported (Ravaris, Nies, Robinson, et al., 1976; Rickels & Case, 1982; Rickels, Case, Weberlowsky, et al., META-ANALYSIS OF PLACEBO http://journals.apa.org/pt/prevention/volume1/pre0010002a.html 3 of 16 3/11/2009 2:51 PM 1981; Robinson, Nies, & Ravaris, 1973). The methods described by Smith et al. (1980) were used to estimate effect sizes for two studies in which means and SDs were not reported. One of these studies (Goldberg, Rickels, & Finnerty, 1981) reported the t value for the pre–post comparisons. The effect size for this study was estimated using the formula: